Avoid manually ending read stream #346
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We do not need to emit 'end' here, it will be called automatically by the node stream implementation. The problem is the following:
Lets assume we have a read stream on a ZIP file, that we want to parse. We use the following construct:
const zip = unzip.Parse();
myStream.pipe(zip);
zip.on('data', data => {
// do something with the data
});
zip.on('end', () => {
// finish processing the data
});
What happens is that zip contains a write stream and a read stream. myStream.pipe(zip) will pipe the myStream stream into zip's write stream, while zip's read stream will be consumed by the 'data' event listener. The 'finish' event pertains to the write stream, while 'end' pertains to the read stream. Thus, with the assumption that finishing the write stream means that the read stream should also finish, as any data that was processed by the write stream before finishing would have resulted in a 'data' event being emitted by the read stream, we emit 'end' when the write stream finishes.
However, the autodrain feature seems to interfere with this assumption. As described in the documentation: 'If you do not intend to consume an entry stream's raw data, call autodrain() to dispose of the entry's contents. Otherwise the stream will halt.' This can cause the following sequence of events:
This effectively causes the reader to not be able to rely on the 'end' event, as more data may come afterwards.